Smart grids promise to improve the efficiency of power grid systems and reduce greenhouse emissions through incorporating power generation from renewable sources and shaping demands to match the supply. Renewable sources include solar or wind. Power generation from these sources is affected by weather factors that can be highly fluctuating. To ensure these energy sources can be utilized efficiently, smart grid systems often shape demand through incentive to match the supply. As a result, the whole system becomes highly dynamic and requires constant adjusting. How to adjust the system can have a great impact on the efficiency and reliability of power grid systems, which offer many opportunities for innovation. In our previous work, we have identified and developed several applications that can be used to optimize power grid operations.
However, these applications rely on the precise estimation of the state of power grid systems. To enable precise estimate of power grid, enormous amount of data from millions of sensors from power grid must be used. Moreover, the relevant data must be delivered to applications within real time constraints. Even though millions of sensors such as phase measurement units (PMU) and smart meters are being widely deployed over the Internet, there does not exist a software system that can collect, store, retrieve, and deliver these amount of data in real time.
Most existing data middleware are either designed for small scale application or built on top of high level system software and APIs. Much indirection is introduced into these systems, which can cause both high overhead and unpredictability due to alternative execution path. For instance, to allow arbitrary insert, deletion, and modifications, the metadata block has to be traversed—and possible several levels of indirection blocks—before data can be accessed.
Other works have been carried out both in research community and commercial world to provide better means to monitor and control power grids. Most of the efforts are targeted towards scalability and Quality of Service (QoS). GridStat is a middleware that provides higher-level abstractions for programmer to develop applications for power grid monitoring. It allows for interoperability across different operating systems, network technologies, programming languages, computer architectures, and even across different vendors' middleware frameworks. It is a specialization of publish-subscribe architecture with use of status variables to provide optimized efficiency. These status variables are provided to publish-subscribers with specified quality of service (QoS) requirements, including timeliness, redundant paths, and computer security. It also provides its interfaces using CORBA, a widely utilized middleware standard. However, GridStat does not provide real time data ingestion and retrieval capability and thus it can only be used for current data instead of historical data.
There are some efforts towards a cyber-enabled energy management system (EMS) and supervisory control and data acquisition (SCADA), with modular components and trustworthy middleware and heterogeneous distributed energy sources (DERs) added to future power grids, which will control and manage energy generation, transmission, and distribution. The information network, including traditional EMS and SCADA, carries out multi-scale networked sensing/processing, communicates distilled information across the grid, and facilitates the closing of a large number of loops in real-time, so that actions are appropriately taken in a timely manner to ensure overall reliability and performance However, those systems cannot scale up to handle millions of sensors as in future smart grids.
More recent work includes the design of large scale data middleware. For instance, Ghemawatt et. al, Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. 2003. The Google file system. SIGOPS Oper. Syst. Rev. 37, 5 (October 2003), 29-43, proposed Google file system, a distributed file system that stress scalability and fault tolerance. Beaver et. al., Doug Beaver, Sanjeev Kumar, Harry C. Li, Jason Sobel, and Peter Vajgel. 2010. Finding a needle in Haystack: facebook's photo storage. In Proceedings of the 9th USENIX conference on Operating systems design and implementation (OSDI'10), 1-8, Vancouver, BC, Canada, 2010, reported Facebook's phone storage and retrieval system that can store petabytes of phone and support millions of stores and retrievals. However, these systems do not provide real time guarantees.
SciDB entails an attempt for a DBMS based solution that would meet the need of scientific uses and increasingly data rich science. It presents specification of a nested array data model based system, which provides “No-overwrite” storage and supports science specific primitive operation as regid and provenance. It uses multidimensional, nested array model with array cells containing records, which in turn can contain components that are multi-dimensional array. It also supports user-defined functions coded in C++, which can be used as an operator on data. These updatable arrays have a history dimension which keeps track of modification history of data to provide provenance. It uses a partitioning which changes over times to support applications where data ingest is not uniform over times—i.e. first partitioning scheme is used for time less than T and second partitioning scheme is used for time greater than T. It also stores self-describing data format so that it can operate on in site data without requiring a load process. SciDB does not tailor its design to power grid data and thus cannot meet the scalability and real time requirements of smart grid applications.